11. Demo: Preparing for Training in Python

Part 1

Training DQN Models: Dataset Preparation

This guide outlines the initial steps for preparing datasets and training DQN models using predefined components.

Key Steps Covered:

1. Dataset Splitting:

   • Load Dataset: Begin by loading the complete dataset into a variable named data.
   • Split Dataset: Divide the dataset into training and test sets following an 80/20 split. Slice out the first 80% for training, leaving the remaining 20% for testing.

2. Verification:

   • Check Splits: Use display functions to ensure no overlap between training and test data. Confirm the correct chronological division between the two sets.

3. Data Type Conversion:

   • Use NumPy: Convert both training and test sets into NumPy arrays of floats using Pandas' value function.

4. Agent Initialization:

   • Define Parameters: Determine the window size to specify periods for input history.

   • Create Agent Instance: Initialize the agent class using the window size and number of features in the data.

Proceed with these preparatory steps to effectively train your DQN model using well-split and formatted datasets.

Part 2

Overview of Key Functions in Training the Agent

This summary outlines the essential functions necessary for building and training an agent through machine learning.

Core Functions Explained:

1. Format Price Function:

   • Ensures correct formatting of prices with a dollar sign.
   • Handles negative values by attaching a negative dollar sign.
   • Limits decimals to two for precision.

2. Sigmoid Function:

   • A fundamental component in machine learning.
   • Helps transform inputs within a specific range.

3. Plotting Trades and Profits:

   • Facilitates visual representation of trading activity.
   • Involves plotting buy signals in red and sell signals in green.
   • Displays price actions and essential indicators such as Bollinger Bands.
   • Marks trades on the graph to observe decision points.

4. Labeling and Observations:

   • Designates x-axis labels for training and testing datasets.
   • Adjusts tick spacing to improve readability.
   • Displays overall cumulative profit or loss for easy analysis.

These steps guide the visualization and performance evaluation of trading strategies, contributing to effective model learning.

Part 3

Understanding Helper Functions for Training and Data Processing

In this guide, the focus is on two essential functions for training models and processing data effectively:

Plotting Training Losses

• Purpose: Visualize training losses using a plot.
• Input: A list of loss values from experience replay.
• Process:
  • Plot the losses.
  • Add titles and labels for clarity.
  • Use plt.show() to display the plot.

Defining the Get State Function

• Purpose: Create a state representation at a given time step.
• Parameters:
  • "t" as the ending index/time step.
  • "n" as the size of the look-back window.
  • Dataset as input.
• Conditions:
  • If data begins within the window size, select corresponding data slice.
  • If not, duplicate initial data to fill the window.

Incorporating Sigmoid Function

• Process:
  • Introduction of a sigmoid function to process feature blocks.
  • Iteration through features and applying sigmoid to transform data.
  • Result stored in a numpy array for further use.

This setup prepares data and functions for effective model training.